1. Field
The present invention relates to illumination compensation in digital images. More particularly, the present invention relates to balancing the dynamic range of a scene within a digital image with contrast enhancement of the scene.
2. Description of Related Art
Image enhancement refers to the science of improving the quality of an image based on some absolute measure. While the problems with image quality are numerous, such as focus problems, depth-of-field issues, motion blur, dynamic range of scene and other sources of noise, the main focus in the digital image processing community has been to improve or enhance fine structure contrasts. This is primarily referred to as contrast enhancement. Contrast enhancement is useful when image contrasts are imperceptible or barely perceptible. Low contrast can be caused by scenes that are hazy or have other poor illumination conditions. This problem is further exacerbated by the fact that cameras with low dynamic range capture the scene with a reduction in dynamic range from the true range present in the scene and this comes with the penalty of further loss of contrast. However, there is experimental evidence that suggests that human perception (subjective quality) is the best when the image's contrast and dynamic range is high. This is the primary motivation for developing a new method for image enhancement.
Contrast enhancement techniques for images are broadly classified into two classes. The first class modifies the intensity histogram of images for contrast enhancement. A special case is the histogram equalization method. See, for example, Anil K. Jain, Fundamentals of Digital Image Processing, Prentice Hall, 1969, pp. 241–244. Histogram equalization applied to an entire image has the disadvantage of attenuating or even removing contrast information of small magnitudes in the scarcely populated histogram regions. This is because neighboring pixels in such regions are mapped to the same output histogram value.
Another method using histogram modification is adaptive histogram equalization. Adaptive histogram equalization applies histogram modification based on local statistics of an image rather than on a global scale. See, for example, R. A. Hummel, “Image Enhancement By Histogram Transformation,” Computer Vision Graphics and Image Processing, vol. 6, 1977, pp. 184–195; R. B. Paranjape, W. M. Morrow and R. M. Rangayyan, “Adaptive-Neighborhood Histogram Equalization For Image Enhancement,” Computer Vision Graphics and Image Processing, vol. 54, no. 3, 1992, pp. 259–267; and, D. Mukherjee and B. Chatterji, “Adaptive Neighborhood Extended Contrast Enhancement And Its Modifications,” Graphical Models and Image Processing, vol. 57, no. 3, 1995, pp. 254–265. Contrast enhancement methods based on adaptive histogram equalization are generally computationally slow. Since such methods operate on purely local statistics, the methods sometime create artifacts that make it hard to distinguish real objects from clutter. There are other variations of adaptive histogram techniques that address the trade-off issues between computational speed and accuracy of enhancement. See, for example, S. M. Pizer, et. al, “Adaptive Histogram Equalization And Its Variations,” Computer Vision Graphics and Image Processing, vol. 39, 1987. pp. 355–368. J. Alex Stark in “Adaptive Image Contrast Enhancement Using Generalizations Of Histograms,” IEEE Transactions on Image Processing, vol. 9, no. 5, May 2000, describes a generalized histogram representation with a goal of reducing the number of parameters to adjust and yet obtain a wide variety of contrast enhancement results.
A second class of contrast enhancement techniques operate directly on the image by applying the principle of separating high and low frequency content in the image. In these cases the image histogram is not adjusted. The frequency contents are instead manipulated and filtered in such a way as to enhance the image contrast. Two examples of this approach are the homomorphic filtering method and the unsharp masking approach. The homomorphic filtering method is described in more detail by A. V. Oppenheim, R. W. Schafer and T. G. Stockham Jr., in “Nonlinear Filtering Of Multiplied And Convolved Signals,” Proc. of IEEE, vol. 56, no. 8, 1968, pp. 1264–1291.
More recent variations of techniques based on the separation of high and low frequency content operate directly on the image based on purely local statistics. See, for example, P. M. Narendra and R. C. Fitch, “Real-Time Adaptive Contrast Enhancement,” IEEE Trans. On Pattern Analysis and Machine Intelligence, vol. 3, no. 6, 1981, pp. 655–661. The local statistics of the image are characterized by the mean and variance in intensity. The mean represents the low frequency content while the variance represents the high frequency content in the image. An adaptive scheme based on these two parameters is used to manipulate the image content so as to enhance contrast in the image. These approaches have also been extended using a multi-scale algorithm as described in A. Toet, “Adaptive Multi-Scale Contrast Enhancement Through Non-Linear Pyramid Recombination,” Pattern Recognition Letters, vol. 11, 1990, pp. 735–742, and K. Schutte, “Multi-Scale Adaptive Gain Control Of IR Images,” Proc. of SPIE, vol. 3661, 1997, pp. 906–914. A similar approach is the ON-OFF filter that was designed as a set of two parallel modules that measures local statistics using the differences of a Gaussian filter. See, for example, S. Grossberg and D. Todorovic, “Neural Dynamics Of 1-D And 2-D Brightness Perception: A Unified Model Of Classical And Recent Phenomena,” Perception and Psychophysics, vol. 43, 1988, pp. 241–277. In this approach, contrast enhancement is realized by combining the outputs of the two modules to improve robustness. The OFF module performs an image inversion on the input image before extracting contrast information.
Prior art methods for contrast enhancement are typically deficient either in the quality of image enhancement due to inherent problems with the method or are computationally slow and require image-specific adjustments of several system parameters. As discussed above, prior art methods based on modification of the intensity histogram of images have the disadvantage of attenuating or even removing contrast information of small magnitudes in the scarcely populated histogram regions. This result occurs because neighboring pixels in such regions are mapped to the same output histogram value. Methods based on adaptive histogram modification attempt to address this problem, but these methods are computationally slow. As also discussed above, other prior art methods operate directly on the image by applying the principle of separating high and low frequency content in the image. In these methods, the image histogram is typically not adjusted. The frequency contents are instead manipulated and filtered in such a way as to enhance the image contrast. These methods are also generally computationally slow and require image specific adjustments to several parameters for improved results. Further, while the contrast enhancement is improved in most cases, the dynamic range of the image is not.
Therefore, there exists a need in the art for enhancing the contrast of a digital image without attenuating or removing contrast information in portions of the image and without requiring significant computation times. There also exists a need in the art for performing such contrast enhancement without requiring image specific adjustments to several parameters to obtain satisfactory results. Finally, there exists a need in the art for controlling dynamic range while contrast enhancement is being performed.